SYNOPSIS
This case study describes a research project to determine the differences between one group of subjects following an agile testing method and another group of subjects following an ad hoc approach to testing. Results showed that the use of an agile approach improved defect detection by some 50% with just a 10% increase in test effort over the control group.
Introduction
My name is Lucjan Stapp and I work as an Assistant Professor at the Warsaw University of Technology. This case study describes research conducted at the Warsaw University of Technology, Faculty of Mathematics and Information Science, to determine the differences between one group employing an agile testing method and a control group using an ad hoc approach to testing.
Overview of the Testing Challenge
Agile approaches are often proposed as a solution to improving a number of characteristics of the testing process; typical improvements are claimed in speed and cost of testing, but many of these claims are only backed up by qualitative or anecdotal evidence.
This research was conducted to try to quantify the benefits of agile versus ad hoc testing approaches.
Teams and Exercises
To check the cost of using test-driven methodology, we have created two groups with the same knowledge of programming in Java:
the test group, later called the TDD team (using test-driven development [TDD] methodology), and
the control group, which, traditionally, does not have unit tests built in.
Both groups have made the same, rather simple applications (about 6 hours of coding for a three-person group).